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ABSTRACT 



With the assistance of learning technology consultants in 
the Technology Teaching and Learning Center (TTLC) at the University of 
Houston-Downtown (Texas) , professors have shifted their paradigms and are 
taking the leap to use more high-risk World Wide Web technologies in their 
courses. One that has become a hallmark is delivering exams via the Internet. 
With advanced authoring tools such as Asymetrix ToolBook II Instructor, 
students can take tests on computer workstations inside computer labs within 
their university, at distant education sites, or at home. This paper 
discusses: (1) planning and preparation, including software selection, 

training, and time management; (2) risks related to question widgets, 
question scoring, Java class files, exam scoring, and Web browser components; 
and (3) high-risk precautions, including security issues and back-up plans. 
Three figures present the Java class location, student log-in, and a 
scoring/tracking screen. (Author/AEF) 
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Abstract: Just as technology trends improve and we approach the next millenium, 
so have our abilities to perceive farther and visualize new methods of testing. With 
the assistance of learning technology consultants in the Technology Teaching and 
Learning Center (TTLC) at University of Houston-Downtown, professors have 
shifted their paradigms and are taking the leap to use more high-risk web 
technologies in their courses. One that has become an instant hallmark is 
delivering exams via the Internet. 
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A new state of consciousness has taken form - the reality that learning can take place outside the 
confines of a classroom. This is not meant only in terms of the ever-evolving distance learning and video 
conferencing, but also by using web technologies as a supplement. 

In one of the nation’s most diverse institutions of higher education, the University of Houston- 
Downtown (UHD) in Houston, Texas, students are grasping ideas and concepts through non-traditional 
teaching methods. For example, faculty members incorporate electronic mail, listservs, and World Wide 
Web home pages into their classes to provide more innovative ways of delivering instruction. Professors 
have the opportunity to incorporate the latest computer technologies into their courses with the assistance 
of learning technology consultants like us who work inside the UHD Technology Teaching and Learning 
Center (TTLC). The TTLC provides resources and support for the university community to explore new 
technologies and capitalize on their potential to improve learning. It offers a wide range of programs, tools, 
and activities that enhance the education and opportunities available to the diverse student body of this 
dynamic university. By including such learning facets, students are ready to venture into the “real world” 
being more confident, prepared, and capable of using technology. They are ready to surpass their 
competitors and end up on top. 

Just as technology trends improve and we approach the next millenium, so have our abilities to 
perceive farther and visualize new methods of delivering instruction. With this in mind, professors at UHD 
have shifted their paradigms and are taking the leap to utilize more high-risk web technologies in their 
courses. One that is becoming an instant hallmark is delivering exams via the Internet. 

The development of authoring tools such as Asymetrix’s ToolBook II Instructor enables more 
professors to break the barriers of test taking. With advanced programs such as this, students can take tests 
on computer workstations inside computer labs within their university, at distant educational sites, or at 
home. 

2. Planning and Preparation 

Throughout the 1997-98 school year at UHD, faculty members took the leap of developing online 
exams to be delivered via the Internet as a supplemental method of giving examinations during 
instructional television (ITV) courses. A great deal of planning and preparation took suit before the initial 
test taking, but ultimately the exams were successfully administered and conducted. 
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After assessment of the current software titles available in the TTLC, Asymetrix’s ToolBook II 
Instructor 5.01 was the software of choice. 

ToolBook II Instructor is a high-end authoring tool designed for the flexible creation of content-rich 
online learning applications. Its wide array of tools and predefined content includes wizards, widgets, and 
templates. It comes with a catalog full of preprogrammed interactive objects that can be dragged and 
dropped into applications. The authoring tools provide for a range of distribution options that are sure to 
meet many users needs. Because it was necessary to deploy over the Internet, we decided to use the HTML 
& Java export option built into the ToolBook II authoring environment. 

Using the ToolBook II “Export for Web” feature, the online exam was converted directly to an 
Internet-ready format (a combination of HTML and Java) for delivery on the World Wide Web. Anyone 
with Internet access, a web browser, and Uniform Resource Locator (URL) of the exam can take the test 
without having to install a plug-in. 

After determining the software application to use, we (the technology consultants) then had to provide 
training for the faculty members. This involved femiliarizing the professors with the information needed to 
create a ToolBook II Instructor Book (examination), which includes learning how to add: 

❖ a title page ❖ navigation widgets 

❖ duplicate pages ❖ an exam summary page 

❖ text fields ❖ a scoring button 

❖ question widgets 

In addition, time management was very important in the development of the online examinations. 
Given that faculty members participate in a number of activities including teaching courses and attending 
conferences, we wanted to make sure they learned the skills necessary to build successful exams. As 
technology consultants, it is our responsibility to train the faculty specifically on a need-to-know basis. 

This included providing one-on-one hands-on training, detailed documentation, and constant support. 

3. The Components and Risks 

During the online exam development process, there were a number of unexpected factors discovered. 
These factors are what allowed us to learn that online testing is indeed a high-risk web technology. 

3.1 Question Widgets 

All of the ToolBook II Instructor 5.01 question widgets are created fi'om ToolBook II objects. Many, 
particularly multiple-choice question widgets, consist of groups of objects in fours. The question widget 
behavior is specified using scripts. If a question widget’s default does not meet the needs of your 
application, you can customize it. The question widget in the Standard Widget Catalog are constructed 
fi'om standard ToolBook II objects, so you can modify them easily if you are comfortable with OpenScript 
programming. However, the question widgets in the Internet Widget Catalog are implemented not with 
OpenScript, but with Java. These question widgets can not be modified in the same way the standard 
question widgets are modified. This proved to be a problem with high-risks involved. For instance, if a 
professor has a multiple-choice test with five answers rather than four, the question widgets (which are pre- 
programmed with four options) would need to be modified using scripting that the professor may not have 
time to learn. 

3.2 Question Scoring 

In one instance, we actually created an exam with 50 questions using a question widget with four 
choices. We manually duplicated one of the choices before setting the question properties for each 
question. So, instead of having only options a, b, c, and d, we had a, b, c, d, and e. After setting the 
scoring properties for each question widget and exporting the test as HTML, we were faced with the 
problem of answer e being scored correctly even if the correct answer was defined as a, b, c, or d in the 
scoring properties. This was discovered when we took the exam and selected e as the answer for each 
question for testing purposes. We found that a solution would be to create the test with four multiple- 
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choice questions. This allowed us to avoid working with any scripting and gave us time to focus on the 
exam development. 

3.3 Java Class Files 

Before the exams could run effectively, ToolBook II Java classes needed to be downloaded from the 
Asymetrix web site (http://www.asvmetrix.com) and then installed in a designated area on the web server 
where the exam would be located. If an exam were accessed during a time when the necessary Java class 
files were not installed on the server, then the test would not fimction properly. 




Figure 1: Screen view of the window one must access in 
ToolBook II Instructor 5.01 in order to indicate the Java Class 
Location. 

3.4 Scoring the Exam 

Scoring properties were setup for the examinations. This was configured in the Instructor’s Book 
Properties. When these settings were defined correctly, it allowed for the scores to be sent to the student. 
When the student entered the exam, he or she was prompted with the login window to enter the correct 
name and email address in order to receive the score. 
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Figure 2: Screen view of the window students must 
fill out so they may receive their scores and so the 
instructor will know is taking the examination. 
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In the Instructor Book properties, there is also an option to send a log to the professor so that he or she 
can find out more information, like the computer machine name fi’om which the student took the exam. 
This is excellent information to have for security purposes. 




Figure 3: Screen view of the Scoring, Tracking window in the 
Instructor Book Properties. 

During an actual examination we discovered that some student logs were not emailed to the professor. 
Though tested continuously, we were still unable to determine the reason why they were not received. 

3.5 Web Browser 

Before deploying the exams over the Internet, it was important to know what computer software the 
students enrolled in the courses had access to at their locations. The ToolBook II Instructor 5.01 
requirements specify that tests should be accessed using the Netscape Navigator Gold 3.01 web browser. 
Through trail and error, we discovered that exams could occasionally be accessed using a lower or higher 
version of the Netscape browser. However, during the actual testing situation we required that students 
access the exams using Netscape Navigator Gold 3.01. 

4. High-Risk Precautions 

4.1 Security Issues 

Before and during the examinations, security was an important issue. The security precautions we 
took included setting restrictions on the time students could access the exams and having a proctor 
available at distant locations during the tests to verify that the students taking the test were actually 
registered in the course. In addition, we set up restrictions which enforced students to provided usernames 
and passwords before entering the online examination web pages. 

4.2 Back-up Plan 

When setting up online exams it's also important for the instructor to have a back-up plan in case the 
technology fails. We prepared for this by instructing the students to bring scan-trons on the exam date and 
enter their answers on them as they selected their answers on the computer. Also, it was important to have 
more computers ready than students registered in the class in case one or more computers experienced 
technical difficulties during the examination. 
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5. Conclusion 



By including such a high-risk web technology as online testing into courses, professors are able to 
evaluate the effectiveness of incorporating this type of testing in their distance learning courses and in turn 
recommend this use of the technology to other faculty members. In our organization, we learned that the 
professors as well as the university and technology department benefit in this process. By addressing and 
accessing the online testing issue through the development of tests created with such products as Asymetrix 
ToolBook II Instructor, individuals see that the worth is invaluable to the entire university from the 
administration to the students. 
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